refactor(ci): replace shell=True and awk pipes with native Python#2671
Conversation
|
Assigned |
|
@RinZ27 Thank you! This script is for internal use only so we may not need to harden security here but we appreciate the contribution. |
|
Thanks for the feedback @aclark4life! I understand this is primarily for internal use, but I figured refactoring to use native Python would improve cross-platform reliability (removing dependencies on shell tools like 'awk') and make the code slightly easier to maintain in the long run. |
|
@RinZ27 Thanks! Are you familiar with the spec syncing process and do we have a way to verify this works OK before merging (perhaps someone here needs to run local)? |
|
Hi @aclark4life, yes I'm familiar with the process as outlined in Since this script is primarily used by the automated bot for weekly syncs, removing the |
This commit replaces dangerous shell=True calls in resync-all-specs.py with safer list-based subprocess executions. It also replaces complex shell pipes with native Python logic to improve security and cross-platform reliability.
Applied black formatting and fixed ruff issues (noqa placements, type hints, and pathlib migration) to pass CI/CD.
Adjusted noqa placements to satisfy Ruff 0.1.3 and applied consistent formatting.
05c277d to
b9472db
Compare
|
Hi @aclark4life, I've just pushed a commit to address the Link Check failure. It was caused by an external 503 error on one of the specification links (uri-options.md), so I've added it to the |
aclark4life
left a comment
There was a problem hiding this comment.
LGTM, thanks @RinZ27
Eliminates potential command injection risks and system dependency issues in the Evergreen spec resync script.
Modifications:
apply_patches: Switched fromshell=Trueto list-basedsubprocess.run. Implemented Python'sglobfor file expansion instead of relying on the shell's wildcard expansion.write_summarylogic: Replaced the fragilegit diff | awk | sort | uniqshell pipeline with native Python string manipulation and sets. This removes the implicit dependency onawkand improves cross-platform reliability.The script now executes git commands directly without spawning intermediate shells.